Prior-based Coregistration and Cosegmentation

We propose a modular and scalable framework for dense coregistration and cosegmentation with two key characteristics: first, we substitute ground truth data with the semantic map output of a classifier; second, we combine this output with population deformable registration to improve both alignment and segmentation. Our approach deforms all volumes towards consensus, taking into account image similarities and label consistency. Our pipeline can incorporate any classifier and similarity metric. Results on two datasets, containing annotations of challenging brain structures, demonstrate the potential of our method.Comment: The first two authors contributed equall

Fast and easy blind deblurring using an inverse filter and PROBE

PROBE (Progressive Removal of Blur Residual) is a recursive framework for blind deblurring. Using the elementary modified inverse filter at its core, PROBE's experimental performance meets or exceeds the state of the art, both visually and quantitatively. Remarkably, PROBE lends itself to analysis that reveals its convergence properties. PROBE is motivated by recent ideas on progressive blind deblurring, but breaks away from previous research by its simplicity, speed, performance and potential for analysis. PROBE is neither a functional minimization approach, nor an open-loop sequential method (blur kernel estimation followed by non-blind deblurring). PROBE is a feedback scheme, deriving its unique strength from the closed-loop architecture rather than from the accuracy of its algorithmic components

Contextual Object Detection with a Few Relevant Neighbors

A natural way to improve the detection of objects is to consider the contextual constraints imposed by the detection of additional objects in a given scene. In this work, we exploit the spatial relations between objects in order to improve detection capacity, as well as analyze various properties of the contextual object detection problem. To precisely calculate context-based probabilities of objects, we developed a model that examines the interactions between objects in an exact probabilistic setting, in contrast to previous methods that typically utilize approximations based on pairwise interactions. Such a scheme is facilitated by the realistic assumption that the existence of an object in any given location is influenced by only few informative locations in space. Based on this assumption, we suggest a method for identifying these relevant locations and integrating them into a mostly exact calculation of probability based on their raw detector responses. This scheme is shown to improve detection results and provides unique insights about the process of contextual inference for object detection. We show that it is generally difficult to learn that a particular object reduces the probability of another, and that in cases when the context and detector strongly disagree this learning becomes virtually impossible for the purposes of improving the results of an object detector. Finally, we demonstrate improved detection results through use of our approach as applied to the PASCAL VOC and COCO datasets

A Compact Linear Programming Relaxation for Binary Sub-modular MRF

We propose a novel compact linear programming (LP) relaxation for binary sub-modular MRF in the context of object segmentation. Our model is obtained by linearizing an $l_1^+$-norm derived from the quadratic programming (QP) form of the MRF energy. The resultant LP model contains significantly fewer variables and constraints compared to the conventional LP relaxation of the MRF energy. In addition, unlike QP which can produce ambiguous labels, our model can be viewed as a quasi-total-variation minimization problem, and it can therefore preserve the discontinuities in the labels. We further establish a relaxation bound between our LP model and the conventional LP model. In the experiments, we demonstrate our method for the task of interactive object segmentation. Our LP model outperforms QP when converting the continuous labels to binary labels using different threshold values on the entire Oxford interactive segmentation dataset. The computational complexity of our LP is of the same order as that of the QP, and it is significantly lower than the conventional LP relaxation

Complexity of Discrete Energy Minimization Problems

Discrete energy minimization is widely-used in computer vision and machine learning for problems such as MAP inference in graphical models. The problem, in general, is notoriously intractable, and finding the global optimal solution is known to be NP-hard. However, is it possible to approximate this problem with a reasonable ratio bound on the solution quality in polynomial time? We show in this paper that the answer is no. Specifically, we show that general energy minimization, even in the 2-label pairwise case, and planar energy minimization with three or more labels are exp-APX-complete. This finding rules out the existence of any approximation algorithm with a sub-exponential approximation ratio in the input size for these two problems, including constant factor approximations. Moreover, we collect and review the computational complexity of several subclass problems and arrange them on a complexity scale consisting of three major complexity classes -- PO, APX, and exp-APX, corresponding to problems that are solvable, approximable, and inapproximable in polynomial time. Problems in the first two complexity classes can serve as alternative tractable formulations to the inapproximable ones. This paper can help vision researchers to select an appropriate model for an application or guide them in designing new algorithms.Comment: ECCV'16 accepte

A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems

International audienceSzeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While these insights remain generally useful today, the phenomenal success of random field models means that the kinds of inference problems that have to be solved changed significantly. Specifically , the models today often include higher order interactions, flexible connectivity structures, large label-spaces of different car-dinalities, or learned energy tables. To reflect these changes, we provide a modernized and enlarged study. We present an empirical comparison of more than 27 state-of-the-art optimization techniques on a corpus of 2,453 energy minimization instances from diverse applications in computer vision. To ensure reproducibility, we evaluate all methods in the OpenGM 2 framework and report extensive results regarding runtime and solution quality. Key insights from our study agree with the results of Szeliski et al. for the types of models they studied. However, on new and challenging types of models our findings disagree and suggest that polyhedral methods and integer programming solvers are competitive in terms of runtime and solution quality over a large range of model types

Computational optical imaging with a photonic lantern

[EN] The thin and flexible nature of optical fibres often makes them the ideal technology to view biological processes in-vivo, but current microendoscopic approaches are limited in spatial resolution. Here, we demonstrate a route to high resolution microendoscopy using a multicore fibre (MCF) with an adiabatic multimode-to-single-mode "photonic lantern" transition formed at the distal end by tapering. We show that distinct multimode patterns of light can be projected from the output of the lantern by individually exciting the single-mode MCF cores, and that these patterns are highly stable to fibre movement. This capability is then exploited to demonstrate a form of single-pixel imaging, where a single pixel detector is used to detect the fraction of light transmitted through the object for each multimode pattern. A custom computational imaging algorithm we call SARA-COIL is used to reconstruct the object using only the pre-measured multimode patterns themselves and the detector signals.This work was funded through the "Proteus" Engineering and Physical Sciences Research Council (EPSRC) Interdisciplinary Research Collaboration (IRC) (EP/K03197X/1), by the Science and Technology Facilities Council (STFC) through STFC-CLASP grants ST/K006509/1 and ST/K006460/1, STFC Consortium grants ST/N000625/1 and ST/N000544/1. S.L. acknowledges support from the National Natural Science Foundation of China under Grant no. 61705073. DBP acknowledges support from the Royal Academy of Engineering, and the European Research Council (PhotUntangle, 804626). The authors thank Philip Emanuel for the use of his confocal image of A549 cells and Eckhardt Optics for their image of the USAF 1951 target. The authors sincerely thank the anonymous reviewers of this paper for their detailed and considered feedback which helped us to improve the quality of this paper significantly.Choudhury, D.; Mcnicholl, DK.; Repetti, A.; Gris-Sánchez, I.; Li, S.; Phillips, DB.; Whyte, G.... (2020). Computational optical imaging with a photonic lantern. Nature Communications. 11(1):1-9. https://doi.org/10.1038/s41467-020-18818-6S19111Wood, H. A. C., Harrington, K., Birks, T. A., Knight, J. C. & Stone, J. M. High-resolution air-clad imaging fibers. Opt. Lett. 43, 5311–5314 (2018).Akram, A. R. et al. In situ identification of Gram-negative bacteria in human lungs using a topical fluorescent peptide targeting lipid A. Sci. Transl. Med. 10, eaal0033 (2018).Shin, J., Bosworth, B. T. & Foster, M. A. Compressive fluorescence imaging using a multi-core fiber and spatially dependent scattering. Opt. Lett. 42, 109–112 (2017).Papadopoulos, I. N., Farahi, S., Moser, C. & Psaltis, D. Focusing and scanning light through a multimode optical fiber using digital phase conjugation. Opt. Express 20, 10583–10590 (2012).Čižmár, T. & Dholakia, K. Exploiting multimode waveguides for pure fibre-based imaging. Nat. Commun. 3, 1027 (2012).Plöschner, M., Tyc, T. & Čižmár, T. Seeing through chaos in multimode fibres. Nat. Photon. 9, 529–535 (2015).Birks, T. A., Gris-Sánchez, I., Yerolatsitis, S., Leon-Saval, S. G. & Thomson, R. R. The photonic lantern. Adv. Opt. Photon. 7, 107–167 (2015).Birks, T. A., Mangan, B. J., Díez, A., Cruz, J. L. & Murphy, D. F. Photonic lantern’ spectral filters in multi-core fiber. Opt. Express 20, 13996–14008 (2012).Edgar, M. P., Gibson, G. M. & Padgett, M. J. Principles and prospects for single-pixel imaging. Nat. Photon. 13, 13–20 (2019).Mahalati, R. N., Yu, Gu. R. & Kahn, J. M. Resolution limits for imaging through multi-mode fiber. Opt. Express 21, 1656–1668 (2013).Amitonova, L. V. & de Boer, J. F. Compressive imaging through a multimode fiber. Opt. Lett. 43, 5427–5430 (2018).Mallat, S. A Wavelet Tour of Signal Processing 2nd edn (Academic Press, Burlington, MA, 2009).Lustig, M., Donoho, D. & Pauly, J. M. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007).Davies, M., Puy, G., Vandergheynst, P. & Wiaux, Y. A compressed sensing framework for magnetic resonance fingerprinting. SIAM J. Imaging Sci. 7, 2623–2656 (2014).Wiaux, Y., Puy, G., Scaife, A. M. M. & Vandergheynst, P. Compressed sensing imaging techniques in radio interferometry. Mon. Not. R. Astron. Soc. 395, 1733–1742 (2009).Carrillo, R. E., McEwen, J. D. & Wiaux, Y. Sparsity averaging reweighted analysis (SARA): a novel algorithm for radio-interferometric imaging. Mon. Not. R. Astron. Soc. 426, 1223–1234 (2012).Katz, O., Bromberg, Y. & Silberberg, Y. Compressive ghost imaging. Appl. Phys. Lett. 95, 131110 (2009).Sun, B., Welsh, S. S., Edgar, M. P., Shapiro, J. H. & Padgett, M. J. Normalized ghost imaging. Opt. Express 20, 16892–16901 (2012).Kim, M., Park, C., Rodriguez, C., Park, Y. & Cho, Y.-H. Superresolution imaging with optical fluctuation using speckle patterns illumination. Sci. Rep. 5, 16525 (2015).Combettes, P. L. & Pesquet, J. -C. in Fixed-Point Algorithms for Inverse Problems in Science and Engineering (Springer, New York, 2011).Komodakis, N. & Pesquet, J.-C. Playing with duality: an overview of recent primal dual approaches for solving large-scale optimization problems. IEEE Signal Proc. Mag. 32, 31–54 (2015).Chandrasekharan, H. K. et al. Multiplexed single-mode wavelength-to-time mapping of multimode light. Nat. Commun. 8, 14080 (2017).Wadsworth, W. J. et al. Very high numerical aperture fibers. Photon. Technol. Lett. 16, 843–845 (2004).Pesquet, J.-C. & Repetti, A. A class of randomized primal-dual algorithms for distributed optimization. J. Nonlinear Convex Anal. 16, 2353–2490 (2015).Chambolle, A., Ehrhardt, M. J., Richtárik, P. & Schönlieb, C.-B. Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications. SIAM J. Optim. 28, 2783–2808 (2018).Bolte, J., Sabach, S. & Teboulle, M. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014).Chouzenoux, E., Pesquet, J.-C. & Repetti, A. A block coordinate variable metric forward-backward algorithm. J. Glob. Optim. 66, 457–485 (2016).Flusberg, B. A. et al. Fiber-optic fluorescence imaging. Nat. Methods 2, 941–950 (2005).Tsvirkun, V. et al. Bending-induced inter-core group delays in multicore fibers. Opt. Express 25, 31863–31875 (2017).Candès, E. J., Wakin, M. B. & Boyd, S. Enhancing sparsity by reweighted l1 minimization. J. Fourier Anal. Appl. 14, 877–905 (2008).Condat, L. A primal–dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158, 460–479 (2013).Vu, B. C. A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comp. Math. 38, 667–681 (2013)